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Analysis  of  earlier  automated  training  development  needs  appeared  con¬ 
strained.  Certain  training  design  limitations  were  imposed  by  a  rapid  system 
development  schedule  for  the  Field  Artillery's  AN/TPQ-37  Firefinder  Radar  and 
its  A17E11  training  device.  A  training  device  was  proposed  to  reduce  costs 
that  would  accrue  if  soldiers  were  trained  only  on  an  operational  system,  where 
numbers  trained  would  be  limited  and  length  of  time  to  train  hard  to  control. 

Such  a  training  device  can  concentrate  attention  and  effort  to  accomplish  a 
better  integrated  instructional  process  with  attainment  of  individual  and  crew 
task  objectives. 
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With  the  accelerated  acquisition  program,  early  training  design  documents 
could  not  fully  accompany  the  radar  and  device.  Development  of  test  and  evalu¬ 
ation  acceptance  programs  was  seriously  curtailed.  These  programs  could  furnish 
training  design  guidelines  if  planned  with  greater  detail  in  human  factors  and 
personnel  support  for  the  actual  equipment  and  device  design  products.  An  inven¬ 
tory  form  for  Firefinder  training  requirements  was  constructed  to  alleviate  part 
of  the  information  constraint.  The  form  was  oriented  on  the  system  training 
device  to  evaluate  the  utility  of  the  commitment  to  simulated  training  and  trans¬ 
fer  to  the  AN/TPQ-37  Radar.  This  form  was  also  the  primary  instrument  for  a 
personnel  test  of  training  and  the  Firefinder  systems  by  verifying  training  design 
specifications  and  prior  results  from  concept  evaluation  and  user  test  phases. 

A  convergence  of  Firefinder  course  design  and  test  requirements  occurred  at 
the  earlier  concept  evaluation  and  user  test  phases  (Lovell  et  al.,  1980)  indi¬ 
cating  two  apparent  conditional  training  constraints.  First,  there  were  limited 
data  available  about  what  should  constitute  testable  training  on  the  A17E11  device 
and  AN/TPQ-37  Radar.  Secondly,  the  user  test  was  compelled  to  base  evaluation  of 
training  and  system  suitability  on  part  of  the  first  conditional  constraint  find¬ 
ings,  while  extracting  training  measures  of  effectiveness  subject  to  continuing 
revisions  in  content  and  performance  standards.  Under  these  conditions  there 
were  training  issues  and  measures  that  were  not  tested  at  a  desired  level  of  pre¬ 
cision.  Student,  instructor,  training  device  and  equipment  system  relationships 
were  not  identified  sutficiently  to  evaluate  which  tasks,  operations  or  system 
features  defined  the  best  test  of  training  system  capability. 

Other  information-gathering  alternatives  were  not  proposed  due  to  the  pres¬ 
sure  of  pending  training  development  and  system  test  schedules.  Neither  an 
analysis  approach  nor  model  could  concurrently  evolve  which  would  more  economi¬ 
cally  "test"  a  small  number  of  operators  or  mechanics  in  a  manner  similar  to  a 


The  views  expressed  in  this  paper  are  those  of  the  authors  and  do  not  necessarily 
reflect  the  view  of  the  U.S.  Army  Research  Institute  or  the  Department  of  the  Army. 
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structured  "test  pilot"  evaluation  (Kratochwill,  1978).  Naturally,  design  of 
performance  criteria  should  begin  with  the  system  creation.  Later  evaluation 
of  personnel  test  requirements  and  training  can  proceed  directly  from  those 
documented  design  guidelines  which  specify  human  factors  and  personnel  require¬ 
ments  for  system  engineering  and  functional  operations.  One  proposed  acquisition, 
test  and  evaluation,  and  training  development  system  has  already  been  demonstrated. 
It  would  design  each  performance  requirement  with  simulator  specifications  and  man- 
machine  interface  controls  (Hritz  &  Purifoy,  Jr.,  1980). 

Thus  a  few  carefully  selected  test-players  could  reliably  exercise  a  system's 
operational  capabilities  and  automated  training  requirements  in  a  completely 
instrumented  scenario,  when  guidelines  specify  human  factors  and  personnel  require¬ 
ments.  Group  test  evaluation  procedures  are  now  relatively  anachronistic  if  meas¬ 
uring  only  operational  task  behaviors.  A  training  device  such  as  the  A17E11  can 
test  a  limited  number  of  personnel  for  both  learning  and  operational  tasks.  But 
it  must  also  display  a  high  degree  of  fidelity,  satisfy  rigorous  design  parameters, 
and  have  full  performance  evaluation  guidelines  for  personnel  test  and  training 
procedures . 

Personnel  training  effectiveness  of  the  training  program  for  the  A17E11 
Fireflnder  radar  trainer  and  AN/TPQ  equipment  was  evaluated  by  an  Interview- 
survey  form  developed  to  examine  training  policy  needs.  This  81  item  form  was 
given  to  53  personnel  selected  as  test-player  subjects.  To  augment  information 
limited  by  the  accelerated  systems  acquisition,  the  form  was  analyzed  as  a  system 
"personnel  test"  by  group,  background,  and  question  (item)  variables  for  learning 
task  effects  on  trainer  A17E11  or  AN/TPQ-37  Radar  training.  Test  subject  responses 
were  used  to  suggest  training  policy  revisions  using  expected  operator  tasks  and 
observed  deficiencies. 

METHOD 

A  questionnaire  approach  to  analyze  training  development  needs  and  personnel 
consequences  is  not  unique.  A  comprehensive  review  format,  however,  was  newly 
formulated  to  recover  performance  objectives  rather  implicitly  expected  in  the 
systems'  design.  There  is  an  innovative  procedure,  additionally,  in  gathering 
and  synthesizing  information  for  course  design  which  was  not  previously  refer¬ 
enced  nor  based  on  immediately  observed  training  conditions.  Moreover,  the 
methodology  application  has  pointed  to  finding  a  further  clarification  and  coherent 
integration  of  system  design  procedures.  Such  procedures  should  project  specified 
training  guidelines  and  personnel  test  requirements  so  that  an  economical  and  accu¬ 
rate  strategy  will  guide  the  parallel  activities  of  Artillery  system  acquisition, 
test  evaluation,  and  training  development. 

If  total  coordination  of  system  design,  test,  and  training  task  objectives  is 
conceptualized  and  implemented,  simulator  and  equipment  systems  should  fully  demon¬ 
strate  any  designed  operational  features.  Any  suggested  modification  data  are, 
then,  still  acceptable  as  system  test,  personnel  and  training  decisions  are  formu¬ 
lated  well  before  system  installation.  To  support  this  adaptive  concept  requires, 
also,  the  early  selection  of  a  centralized  coordinator  to  direct  and  monitor  every 
critical  aspect  of  acquisition,  test  and  training  requirements  to  deliver  effective 
decisions  for  system  design  and  training.  A  coordinator  must  possess  the  stated 
responsibility  to  intercede  anytime  to  effect  the  required  decision  processing  of 
either  institutional  managers,  technical  experts  or  contractual  support  personnel. 
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An  alternative  analysis  approach  was  prompted  by  events  to  bring  a  degree  of 
synthesis  between  course  design  objectives  and  test  sanctions.  This  would  better 
accommodate  training  and  clarify  test  results  for  any  course,  device,  or  equipment 
changes.  Questionnaire  acceptance  suggests  that  the  personnel  test  instrument  was 
effectively  constructed  to  describe  student  concerns  and  equipment  system  relation¬ 
ships.  These  features  were  noted  from  technical  observations  later  verified  by  the 
Firefinder  training  device  item  responses  given  by  students  and  instructors  and  a 
critique  of  technical  reviewers.  Though  a  "one-time"  instrument,  the  design 
review  format  for  training  analysis  may  suggest  some  type  of  standardized 
approach  by  which  critical  training  issue  and  equipment  capability  measures 
can  move  toward  increased  utility  and  precision.  Other  alternative  analysis 
approaches  will  surely  evolve  for  training  device  acquisition  and  training 
development  as  more  advanced  computerized  training  systems  are  requested. 

As  an  example  of  possible  generic  dimensions  following  from  the  question¬ 
naire  structure,  some  standard  content  features  examining  the  A17E11/AN/TPQ-37 
systems  were  projected.  These  generic  dimensions  developed  on  the  given 
systems  were  then  applied  tentatively  in  an  evaluation  of  the  A17E14  Firefinder 
Maintenance  Trainer.  Where  write-in  or  interview  comments  appeared  very  briefly 
because  of  the  highly  detailed  survey  analysis,  there  is  no  suggestion  to  pursue 
collection  of  observer  or  test  player  remarks  during  test  operations,  except  as 
an  analyst  may  wish  to  annotate  some  condition. 

An  Interim  analysis  Instrument  as  the  method  advocated  in  this  report 
could  yield  significant  training  design  information  for  review  of  course  con¬ 
tent  and  simulated  performance  criteria.  When  training  Information  documents 
may  have  omitted  certain  simulated  and  prime  system  training  and  instructional 
guidelines  during  accelerated  development,  an  auxiliary  effort  is  justified. 

That  effort  should  construct  a  training  inventory  and  interview  form  to  obtain 
the  best  personnel  test  data  available.  Developing  a  flexible  questionnaire 
format  to  interpret  user/operator  transactions  with  an  automated  system  (Berger  & 
Hawkins,  1979),  furnishes  a  viable  alternative  to  support  ISD  system  acquisition 
and  training  development  activities  under  constrained  conditions.  This  approach 
is  illustrated  in  that  simulated  training  device/equipment  operations  and  person¬ 
nel  training  needs  were  effectively  augmented  for  the  Firefinder  Radar  Systems. 

A  progressive  review  of  system  acquisition  and  training  design  testing  would 
be  conducted  by  applying  the  instrument  results  using  interface  perceptions  of 
instructor,  student  and  training  device/equipment.  Evolving  content  and  form 
design  questions  which  arose  intimated  a  possible  instrument  combining  eventually 
personnel  test  and  task  inventory  capabilities. 

Research  questions  explored  completion  of  performance  objectives,  proficient 
trainer  transfer  to  the  AN/TPQ-37,  tasks  trained  and  deficiencies.  These  questions 
were  designed  to  generally  answer  whether  this  personnel  test  of  the  implemented 
training  system  could  better  integrate  trainer  (A17E11)  performance  in  the  actual 
MOS  13R10  course.  Group  responses  could  indicate  significant  preferences  for 
training  policy  activities,  course  content,  proficiency  needs,  and  augment  already 
proven  systems.  Minimal  background  variables  might  affect  responses  on  training 
performance  standards  while  suggesting  remedial  training  tasks.  Though  operator 
skills  may  be  perceived  as  difficult  to  learn,  tasks  were  to  be  identified  for  a 
revised  task  sequence  and  correct  operational  procedures  to  achieve  proficient 
skill  within  critical  learning  times. 
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RESULTS  AND  DISCUSSION 


Research  questions  stated  for  the  personnel  testing  of  the  training 
effectiveness  and  transfer  in  the  Firefinder  course,  resulted  in  a  generally 
positive  set  of  findings  for  training  design  activities  and  A17EI1/AN/TPQ-37 
operations.  These  questions  were  intended  as  goals  by  which  to  analyze  pro¬ 
gressive  achievement  in  training  development.  They  have  suggested  modifi¬ 
cations  to  support  continuing  training  competence  and  course  Improvements. 
Questionnaire  evidence  and  intensive  two-year  observations  by  the  researchers 
tentatively  found  that  probable  training  effectiveness  for  student  operators 
could  be  fully  expected.  Certain  course  insights  and  Implied  modifications 
can  work  to  furnish  an  optimal  training  program.  Improved  training  develop¬ 
ment  and  device  requirements  were  being  defined  during  the  implementation 
phase  of  trainer  acquisition  and  Instruction.  Questionnaire  items  were 
analyzed  by  percentages  and  the  chi  square  test  of  significance  (.05  level) 
examined  for  each  of  the  item  cross  tabulations  with  associated  correlations. 

Performance  Objectives.  Questionnaire  items  affecting  performance  objectives 
were  analyzed  noting  whether  these  items  would  describe  learning  constraints 
or  options  to  choose  an  effective  training-task  solution.  Operators  are 
expected,  it  appeared,  to  attain  or  exceed  performance  objectives  for  the 
A17E11/AN/TPQ-37  systems  when  complementary  tasks  are  explained,  course  content 
is  made  pertinent,  and  instructor  skills  are  evident.  Items  67,  60,  and  50 
were  interpreted  as  specifically  conveying  the  confirmed  findings  for  the  first 
research  question.  Performance  objectives  in  course  achievement  could  then  be 
further  attained  or  exceeded  as  given  tasks  and  operations  were  exercised  in 
the  proper  sequence.  Operators  succeeded,  responses  indicated,  as  instructors 
displayed  necessary  skills  and  helped  students  on  the  trainer  and  equipment, 
referring  to  manuals  and  radar  experience.  Students  learned  faster  and  better 
utilized  study  time  to  complete  performance  objectives,  responses  agreed,  when 
the  training  sequence  applied  the  best  mix  of  trainer/equipment  practice  and 
study  materials.  Item  38  reflected  a  relatively  conclusive  overview  with  98% 
of  the  test  subjects  significantly  acquiring  "reasonably  to  very  sufficient 
skill"  on  the  trainer  to  operate  the  actual  equipment.  This  finding  additionally 
reinforces  the  cumulative  transfer  evidence  given  below. 

Trainer  Equipment  Transfer.  Proficient  trainer  performance  was  expected  from 
responses  to  transfer  to  the  AN/TPQ-37  Radar  and  result  in  successful  operation. 
Test  subjects  answered  item  30  by  a  significant  majority  (76%)  agreeing  to  the 
performance  similarity  of  the  systems  and  procedures.  Where  personal  background 
of  the  test  students  showed  some  significant  differences,  this  majority  observa¬ 
tion  was  still  upheld.  A  contrast  on  item  24  was  shown  by  the  effects  of  group 
background  differences.  Here  differences  were  experienced  by  the  test  personnel 
in  that  their  "  'make-up'  study  to  reach  required  proficiency  standards"  reflected 
some  individual  training  preferences  and  course  flexibility.  A  compressed  training 
schedule  seemed  to  affect  the  responses  to  item  68  regarding  whether  training  on 
the  actual  system  was  more  effective  and  useful  than  on  the  trainer.  Responses 
tended  to  favor  training  on  the  actual  equipment  which  may  simply  capture  the  pre¬ 
ference  of  the  test  players  preparing  for  their  test  site.  Also  the  interesting 
conclusion  is  implied  that  test  students  had  enough  short-term  training  experience 
to  compare  system  experiences  and  then  prefer  AN/TPQ-37  training  over  initial 
A17E11  training.  The  group  response  to  item  57  showed  about  74%  expecting  to  need 
AN/TPQ-37  proficiency  training  "monthly"  or  more  often.  Researcher  observations 
were  used  to  analyze  this  relationship  suggesting  subjects  were  significantly 
aware  of  A17E1 l/AN/TPQ-37  transfer  skills  needing  practice  in  the  unit  location 


to  complement  resident  training.  Transfer  rrom  the  A17E11  to  AN/TPQ-37 
was  facilitated  by  proficient  map-reading  and  radar  skills,  It  was  noted, 
and  may  be  most  handicapped  If  a  student  has  low  reading  skill  and  below 
average  mental  ability  (Item  81). 

Tasks  Trained  and  Deficiencies.  A  narrative  for  training  performance  stand¬ 
ards  described  in  terms  of  Items,  what  was  trained  effectively  and  deficiencies 
needing  further  training  development  according  to  nine  content  factors.  Review 
of  item  responses  permitted  an  evaluation  that  the  course  development  process 
had  succeeded  in  designing  critical  performance  sequences.  Guidance  furnished 
from  this  process  was  used  to  adjust  proficiency  standards  in  reference  to  prior 
device  acquisition  and  development  requirements.  Training  of  critical  tasks  and 
identifying  deficiencies  were  predicated  on  relating  other  items  (43,  46,  and 
54)  for  example,  using  group  difference  and  background  variable  difference.  The 
content  factor  results  gave  a  unifying  perspective,  while  a  research  question 
analysis  probed  other  item  relationships  affecting  control  of  training  perfor¬ 
mance  effects.  If  instructors  explained  task  differences  and  assured  availability 
of  training  materials  and  feedback  evaluation  of  student  errors  with  increasing 
efficiency,  answers  agreed,  a  firm  basis  was  prepared  to  control  critical  task 
learning  and  correct  deficiences.  In  spite  of  some  background  variable  differences 
for  item  32,  a  significant  consensus  was  still  obtained  to  report  complete  enough 
"field  training  to  learn  the  required  operational  tasks  for  the  AN/TPQ-37." 

Certain  deficiencies  were  experienced  relating  to  time  in  the  primary  MOS,  time 
in  the  Army,  and  rank.  Item  34  gave  an  overview  evaluation  for  A17E11  task  training 
and  guidance  with  nearly  100%  of  the  test  students  answering  that  "usually  to  com¬ 
pletely  adequate"  monitoring  of  student  errors  was  given  to  direct  feedback  and  cor¬ 
rection. 

Trainer-Course  Testing.  Training  effectiveness  testing  (Finley  &  Strasel,  1978) 
of  the  Flreflnder  course  increased  the  understanding  of  trainer  features  and 
learning  tasks,  respondents  agreed,  to  better  integrate  it  in  the  course  delivery 
and  with  the  AN/TPQ-37  Radar  system.  The  related  research  question  was  confirmed 
by  a  number  of  associated  findings.  Training  requirements  for  the  A17E11  were 
studied  more  (item  47),  replies  agreed,  as  the  course  was  improved  by  on-going 
training  design  changes.  It  was  conceded  (item  73)  that  the  instructor-student 
ratio  of  1  to  6  should  approach  1  to  3  to  increase  the  attention  level  and  interest. 
That  some  instructor-console  tasks  could  require  most  of  the  instructor’s  time  (item 
69)  was  largely  rejected  by  test-player  answers,  but  less  so  the  longer  away  from 
school  radar  training.  The  instructor-console  function  needed  further  development, 
it  appeared,  to  maximize  student  A17E11  simulation  activities.  Generally  test  sub¬ 
jects  significantly  observed  the  effective  A17E11  "course  sequencing"  with  about 
75%  replying  they  were  able  to  make  suggestions  improving  A17E11  instruction  (item 
26)  and  instructors  were  more  often  able  to  answer  A17E11  questions  to  maintain  the 
training  progress  (item  40) .  The  positive  evidence  presented  for  the  other  research 
questions  above  was  also  accepted  as  reasonable  support  for  a  positive  answer  to 
this  last  question  area. 

In  summary  findings  indicated  modifications  acceptable  to  continuing  training 
program  development  for  the  A17E11/AN/TPQ-37  systems.  The  personnel  test  more 
clearly  described  how  the  training  program  could  maximize  the  already  engineered 
potential  for  trainer/radar  training  effectiveness  and  transfer  features.  Training 
policy  decisions  were  derived  from  research  observations  sampling  performance  stand¬ 
ards.  Support  was  provided  for  an  improved  training  design  and  device  acquisition 
process  at  generic  and  system  specific  levels. 
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